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DETAILED ACTION 
Response to Amendment 

The amendments filed on 01/12/2004 have been entered. Claims 1, 9 and 16 have been amended: 
Claims 2, 12, 19-20, 22-23, and 25-26 have been canceled. Claims 1, 3-1 1, 13-18, 21, 24, 27-28 
are pending in the application. 

Claim Rejections - 35 USC § 103 

1 The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

2. Claims 1, 3-11, 13-18, 21, 24, 27-28 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Bradski et al. U.S. Pat. No. 6,363,160 (hereinafter Bradski) in view of Zabih et 
al. U.S. Patent No. 6,181,817 (hereinafter Zabih), Kenet et al. U.S. Patent No. 5,016, 173 
(hereinafter Kenet) and Darrell et al. U.S. Patent No. 6,188,777 (hereinafter Darrell). 



3. Claim 1: 

(1) Bradski teaches an automated calibration system (figure 1; column 3, lines 34-55) to 
track a selected object through a series of frames of data (column 7, lines 55-65), comprising: 

A display device (e.g., figure 1 1) to display at least one image frame received from an 
image input device, wherein the image frame includes a calibration rectangle (column 4, lines 2 
60; column 15, lines 13-58; figures 4, 10A and 10B); 
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An image selection device to select, via the calibration rectangle, the selected object in 
the at least one image frame (column 3, lines 59-67, column 4, lines 1-10, and column 7, lines 
33-54; column 15, lines 13-58); 

An image source device (video source 1 102 of figure 1 1) to provide a hue saturation 
value (HSV) data array of pixels forming the at least one image frame (column 4, lines 2-57; 
column 13, lines 45-52; column 15, lines 13-58; figures 3A and 3B); and 

A processing device (figures 1-9; column 13, lines 55-67; column 14, lines 1-59) to 
determine analysis data for pixels (column 4, lines 2-57; figures 4, 3 A and 3B) within the 
calibration rectangle (column 5, lines 52-67; column 6, lines 1-44; column 15, lines 13-58; 
figures 4, 10A and 10B), based on the HSV data array (column 4, lines 2-57; figures 3A and 3B), 
and determine test analysis data (e.g., window sizing parameters) for a set of adjacent test 
windows (e.g., search windows or boxes or windows of FIG. 14A displaying images; column 5, 
lines 52-67; column 6, lines 1-44; column 15, lines 13-58), each of the adjacent test windows 
(e.g., the moving search windows) having a same shape and a same pixel size (e.g., column 7; 
column 14-15) as the calibration rectangle (e.g., the entire video frame or a calculation region in 
a form of rectangle or a search window; column 7, lines 5-55; column 15, lines 13-58), wherein 
tracking data, to track the selected object (e.g., the selected gestures), is selected from one of the 
calibration rectangle (e.g., the calculation region in a form of rectangle) and the adjacent test 
windows (e.g., the search windows) having a highest tracking probability (i.e., the tracking data 
such as window location parameters are determined/adjusted and a search window having the 
largest connected region of a probability distribution and the greatest probability density is 
selected, column 6, lines 45-67; column 7, lines 55-67; column 8, lines 1-9; column 15, lines 13- 
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58) and each of the adjacent test windows share at least one pixel with the calibration rectangle 
(e.g., column 7, lines 5-55; column 15, lines 13-58). 

(2) However, Bradski is silent on the claim limitation of "lowest sum of a hue standard 
deviation and a saturation standard deviation". 

(3) Darrell in view of Zabih and Kenet teaches selecting an object based on the "lowest 
sum of a hue standard deviation and a saturation standard deviation." (e.g., Darrell column 10, 
lines 44-65; Zabih column 2, lines 20-37; Kenet column 20, lines 10-30; Kenet column 16, lines 
1-20). Darrell teaches selecting a tracking object based on the optimum ranges and the highest 
probability density of a search region by collecting the variance data of joint probability 
distributions for the observed color data while Zabih teaches selecting a best match result (with 
the optimum ranges and the highest probability density) for an object based on the color joint 
histograms of the candidate images and based on the calculation of the standard deviations 
(variance) for each color in HSV color space while Kenet teaches finding a search region based 
on the optimum ranges/criteria by optimization and joint histograms. 

(4) It would have been obvious to one of ordinary skill in the art to have incorporated the 
Darell/Zabih/Kenet's selection means with the optimum ranges/criteria into the Bradski's 
invention to select a tracking object based on the lowest sum of a hue standard deviation and a 
saturation standard deviation. Bradski teaches performing pattern recognition based on Gaussian 
and Chi-square distributions to determine a best match for the tracked object and therefore 
suggests selecting a tracking object based on the lowest sum of a hue standard deviation and a 
saturation standard deviation because such a selection of optimum ranges yields the best match 
tracking object. 
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(5) One having the ordinary skill in the art would have been motivated to do this because 
it would have provided a routine experimentation of the optimum ranges and criteria for object 
selection to find the best match tracking object. 



Claim 3 : 

The claim 3 encompasses the same scope of invention as that of claim 1 except additional 
claimed limitation of a digital camera. However, Bradski further discloses the claimed limitation 
of the digital camera (Bradski column 4, lines 1-57), 

Claim 4: 

The claim 4 encompasses the same scope of invention as that of claim 1 except additional 
claimed limitation of a single device. However, Bradski further discloses the claimed limitation 
of the single device (figure 1). 

Claim 5: 

The claim 5 encompasses the same scope of invention as that of claim 1 except additional 
claimed limitation of the processing device calculating a mean hue and a standard deviation of a 
hue of the pixels representing the selected object. However, Bradski further discloses the claimed 
limitation of the processing device (figures 1- 9) calculating a mean hue and a standard deviation 
of a hue of the pixels representing the selected object (figures 3 A, 3B and 4; column 4, lines 2- 
57, column 10, lines 1-16, column 5, lines 31-63, and column 6, lines 22-44; column 12, lines 1- 
67). 

Claim 6: 
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The claim 6 encompasses the same scope of invention as that of claim 5 except additional 
claimed limitation of the selected object not being tracked if the mean hue or the standard 
deviation of the hue is less than predetermined levels. However, Bradski further discloses the 
claimed limitation of the selected object not being tracked if the mean hue or the standard 
deviation of the hue is less than predetermined levels (It is apparent that a Gaussian or Chi- 
Square hue probability distribution of Bradski involves the mean hue or the standard deviation of 
the hue and search for the area with the greatest probability density involves the motion-tracking 
processing of not tracking a region where the mean hue or the standard deviation of the hue 
being less than predetermined levels; figures 3 A, 3B and 4; column 4, lines 2-57, column 10, 
lines 1-16, column 5, lines 1-63, and column 6, lines 22-44; column 8, lines 10-38; column 12, 
lines 1-67). 

Claim 7: 

The claim 7 encompasses the same scope of invention as that of claim 1 except additional 
claimed limitation of the analysis module calculating a mean saturation and a standard deviation 
of a saturation of the pixels representing the selected object. However, Bradski further discloses 
the claimed limitation of the processing device (e.g., figures 1-9) calculating a mean saturation 
and a standard deviation of a saturation of the pixels (e.g., a mean saturation and a standard 
deviation of a saturation of the pixels is an inherent property of a Gaussian or Chi-Square 
probability density) representing the selected object (figures 3 A, 3B and 4; column 4, lines 2-57, 
column 10, lines 1-16, column 5, lines 31-63, and column 6, lines 22-44; column 8, lines 10-38; 
column 12, lines 1-67). 

Claim 8: 
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The claim 8 encompasses the same scope of invention as that of claim 7 except additional 
claimed limitation of the colored object not being tracked if the mean saturation or the standard 
deviation of the saturation is less than predetermined level. However, Bradski further discloses 
the claimed limitation of the selected object not being tracked if the mean saturation or the 
standard deviation of the saturation is less than predetermined level (figures 3 A, 3B and 4; 
column 4, lines 2-57, column 10, lines 1-16, column 5, lines 1-63, and column 6, lines 22-44; 
column 8, lines 10-38; column 10, lines 35-67; column 12, lines 1-67). 

4. Claim 9: 

The claim 9 encompasses the same scope of invention as that of claim 1 in a method 
form. The claim is rejected for the same reason as set forth in claim L 
Claim 10: 

The claim 10 encompasses the same scope of invention as that of claim 9 except 
additional claimed limitation of converting a pixel data array for the at least one image frame 
from a red-green-blue color space (RGB) data array to the HS V data array. However, Bradski 
further discloses the claimed limitation of converting a pixel data array for the at least one image 
frame from a red-green-blue color space (RGB) data array to the HSV data array (column 4, 
lines 1-57). 

Claim 11: 

The claim 1 1 encompasses the same scope of invention as that of claim 28 except 
additional claimed limitation of applying the pixel data from an entire frame to the pixel- 
classification look-up map wherein if the amount of the pixels associated with the selected object 
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is greater than a predetermined amount, the calibration method restarts. However, Bradski 
further discloses the claimed limitation of applying the pixel data from an entire frame to the 
pixel-classification look-up map (column 4, lines 45-60) wherein if the amount of the pixels 
associated with the selected object is greater than a predetermined amount, the calibration 
method restarts (figures 3 A, 3B and 4; column 4, lines 1-61; column 6, lines 5-57; column 10, 
lines 35-67). 

Claim 13: 

The claim 13 encompasses the same scope of invention as that of claim 10 except 
additional claimed limitation of thresholding the HSV data array of pixels and disregarding pixel 
data for each of the pixels having a product of a saturation coordinate and a value coordinate 
below a predetermined threshold amount. However, Bradski further discloses the claimed 
limitation of thresholding the HSV data array of pixels and disregarding pixel data for each of 
the pixels having a product of a saturation coordinate and a value coordinate below a 
predetermined threshold amount (From the Bradski' s teaching that "Hue values are only 
accumulated if their corresponding S and V values are above respective S and V thresholds". . it 
can be seen that pixel data with the product of S and V values less than a threshold would be 
excluded from accumulation to the Hue values. See figures 3 A, 3B and 4; column 4, lines 1-57; 
column 5, lines 1-61; column 10, lines 35-67; column 13, lines 55-67; column 14, lines 1-59). 

Claim 14: 

The claim 14 encompasses the same scope of invention as that of claim 10 except 
additional claimed limitation of calculating a mean hue and a standard deviation of a hue of the 
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pixels in the selected object. However, Bradski further discloses the claimed limitation of 
calculating a mean hue and a standard deviation of a hue of the pixels (The mean hue and 
standard deviation of a hue values are inherently associated with hue probability density) in the 
selected object (column 4, lines 2-57, column 10, lines 1-16, column 5, lines 31-63, and column 
6, lines 22-44; column 8, lines 10-38; column 10, lines 35-67; column 12, lines 1-67). 
Claim 15: 

The claim 15 encompasses the same scope of invention as that of claim 14 except 
additional claimed limitation of restarting the calibration method if the mean hue or the standard 
deviation of the hue is less than predetermined level. However, Bradski further discloses the 
claimed limitation of restarting the calibration method (restarting searching) if the mean hue or 
the standard deviation of the hue is less than predetermined level (figures 3 A, 3B and 4; column 

4, lines 2-57, column 10, lines 1-16, column 5, lines 31-63, and column 6, lines 1-44; column 8, 
lines 10-38; column 10, lines 35-67; column 12, lines 1-67). 

Claim 16: 

The claim 16 encompasses the same scope of invention as that of claim 10 except 
additional claimed limitation of calculating a mean saturation and a standard deviation of a 
saturation of the pixels in the selected object. However, Bradski further discloses the claimed 
limitation of calculating a mean saturation and a standard deviation of a saturation of the pixels 
in the selected object (figures 3 A, 3B and 4; column 4, lines 2-57, column 10, lines 1-16, column 

5, lines 31-63, and column 6, lines 22-44; column 12, lines 1-67). 

Claim 17: 
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The claim 17 encompasses the same scope of invention as that of claim 16 except 
additional claimed limitation of restarting the calibration method if one of the mean saturation 
and the standard deviation of the saturation is less than predetermined level. However, Bradski 
further discloses the claimed limitation of restarting the calibration method if one of the mean 
saturation and the standard deviation of the saturation is less than predetermined level (figures 
3 A and 3B; column 4, lines 2-57, column 10, lines 1-16, column 5, lines 31-63, and column 6, 
lines 22-44; column 12, lines 1-67). 

Claim 18: 

The claim 18 encompasses the same scope of invention as that of claim 10 except 
additional claimed limitation of allowing the user to select the selected object. However, Bradski 
further discloses the claimed limitation of allowing the user to select the selected object (column 
4, lines 1-57; column 15, lines 1-67). 

Claim 21: 

The claim 21 encompasses the same scope of invention as that of claim 1 except 
additional claimed limitation of a thresholding module to disregard pixel data for each of the 
pixels having a product of a saturation coordinate and a value coordinate below a predetermined 
threshold amount. However, Bradski further discloses the claimed limitation of a thresholding 
module to disregard pixel data for each of the pixels having a product of a saturation coordinate 
and a value coordinate below a predetermined threshold amount (figures 3 A, 3B, 4-5; column 4, 
lines 1-57; column 13, lines 55-67; column 14, lines 1-59). 

Claim 24: 
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The claim 24 encompasses the same scope of invention as that of claim 1 except 
additional claimed limitation of each of the adjacent test windows having at least one pixel 
overlapping with the calibration rectangle. However, Bradski further discloses the claimed 
limitation of each of the adjacent test windows having at least one pixel overlapping with the 
calibration rectangle (figures 4, 10A-10B and 14A; column 6, lines 1-67; figures 11-14, column 
15, lines 1-67). 

Claim 27: 

The claim 27 encompasses the same scope of invention as that of claim 9 except 
additional claimed limitation of each of the adjacent test windows having at least one pixel 
overlapping with the calibration rectangle. However, Bradski further discloses the claimed 
limitation of each of the adjacent test windows (the moving search windows) having at least one 
pixel overlapping with the calibration rectangle (figures 4, 10A-10B; column 6, lines 1-67; 
figures 11-14, column 15, lines 1-67). 

Claim 28: 

The claim 28 encompasses the same scope of invention as that of claim 9 except 
additional claimed limitation of creating a pixel-classification look-up map for the HSV data 
array of pixels, and the pixel classification map classifies the pixels belonging to the selected 
object based on a hue and a saturation of the pixels. However, Bradski further discloses the 
claimed limitation of creating a pixel-classification look-up map for the HSV data array of 
pixels, and the pixel classification map classifies the pixels belonging to the selected object 
based on a hue and a saturation of the pixels (figures 4, 10A and 10B; column 4, lines 2-57; 
column 6, lines 5-35). 
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Remarks 

5. Applicant's arguments, filed 01/12/2004, paper number 8, have been fully considered but 
they are not deemed to be persuasive. 

6. In Remarks, Applicant argues in essence with respect to claim 1 and similar claims that: 
On page 10, Applicant argues, "Bradski does not disclose, teach, or suggest use of 
adjacent test windows, each having a same shape and a same pixel size as the calibration 
window." 

This is not found persuasive because as set forth in this Office Action, Bradski teaches 
determining test analysis data (e.g., window sizing parameters) for a set of adjacent test windows 
(e.g., search windows or boxes or windows of FIG. 14A displaying images; column 5, lines 52- 
67; column 6, lines 1-44; column 15, lines 13-58), each of the adjacent test windows (e.g., the 
moving search windows) having a same shape and a same pixel size (e.g., column 7; column 14- 
15) as the calibration rectangle (e.g., the entire video frame or a calculation region in a form of 
rectangle or a search window; column 7, lines 5-55; column 15, lines 13-58), wherein tracking 
data, to track the selected object (e.g., the selected gestures), is selected from one of the 
calibration rectangle (e.g., the calculation region in a form of rectangle) and the adjacent test 
windows (e.g., the search windows) having a highest tracking probability (i.e., the tracking data 
such as window location parameters are determined/adjusted and a search window having the 
largest connected region of a probability distribution and the greatest probability density is 
selected, column 6, lines 45-67; column 7, lines 55-67; column 8, lines 1-9; column 15, lines 13- 
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58) and each of the adjacent test windows share at least one pixel with the calibration rectangle 
(e.g., column 7, lines 5-55; column 15, lines 13-58). 

Although the pixel size of the search windows (which meet the claim limitation of 
"adjacent test windows") is not shown the same as the pixel size of the calculation region (which 
meets the claim limitation of "calibration rectangle") in figures 10A, 13A-14E, Bradski teaches 
that the initial search window size and location can be selected to be the same size as the entire 
video frame and the calculation region for tracking is set to be the entire video frame (See 
column 7, lines 10-20). When the search window size and the calculation region size are both set 
to be the entire video frame, the pixel size of the search windows is the same as that of the 
calculation region. Bradski clearly meets the newly added claim limitation and the claim 1 as 
amended is obvious over Bradski/ Zabih/Kenet/Darrell. 

7. In Remarks, Applicant argues in essence with respect to claim 1 and similar claims that: 
On page 1 1, Applicant argues, "Therefore, applicants respectfully submit that such boxes 
are not adjacent test windows as recited in independent claim 1, as amended. Moreover, 
applicants submit that even assuming arguedo that those boxes were adjacent test 
windows, they do not have both the same size and shape as a calibration rectangle." 

This is not found persuasive because Bradski' s calculation region meets the claim limitation of 
"calibration rectangle". The calculation region can be the entire video frame having a 
shape of a rectangle and is used for tracking in video sequences. Therefore, the 
calculation region is clearly a calibration rectangle. Bradski' s search windows meet the 
claim limitation of "adjacent test windows " The search windows of Bradski can be made 
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the same size as the entire video frame (window) wherein the test video frames are shown 
in successive frames ( adjacent frames or adjacent windows) in video sequences (Figures 
14A-14E) and the image within one of the boxes is selected. Furthermore, Applicant 
clearly has implicitly defined "adjacent test windows" in the amended claims. Therefore, 
the term "adjacent test window" is interpreted as the search windows of Bradski which 
are the test video frames or test video windows as shown in successive frames or 
windows and some are adjacent to each other in video sequences (See Figures 14A-14E). 



Conclusion 

8. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1 .136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 

9. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jin-Cheng Wang whose telephone number is (703) 605-1213. 
The examiner can normally be reached on 8:00 AM - 4:30 PM. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mike Razavi can be reached on (703) 305-4713. The fax phone numbers for the 
organization where this application or proceeding is assigned are (703) 308-6606 for regular 
communications and (703) 308-6606 for After Final communications. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is (703) 395-3900. 



jew 

January 27, 2004 




